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Abstract — The problem of sampling a discrete-time sequence 
of spatially bandlimited fields with a bounded dynamic range, in a 
distributed, communication-constrained, processing environment 
is addressed. A central unit, having access to the data gathered 
by a dense network of low-precision sensors, operating under 
stringent inter-node communication constraints, is required to 
reconstruct the field snapshots to maximum accuracy. Both deter- 
ministic and stochastic field models are considered. For stochastic 
fields, results are established in the almost-sure sense. The 
feasibility of having a flexible tradeoff between the oversampling 
rate (sensor density) and the analog-to-digital converter (ADC) 
precision, while achieving an exponential accuracy in the num- 
ber of bits per Nyquist-interval per snapshot is demonstrated. 
This exposes an underlying "conservation of bits" principle: 
the bit-budget per Nyquist-interval per snapshot (the rate) can 
be distributed along the amplitude axis (sensor-precision) and 
space (sensor density) in an almost arbitrary discrete-valued 
manner, while retaining the same (exponential) distortion-rate 
characteristics. An achievable information scaling law for field 
reconstruction over a bounded region is also derived. With A*' 
one-bit sensors per Nyquist-interval, 0(log A'^) Nyquist-intervals, 
and total network bitrate Rnet = ©((log^)^) (per-sensor bitrate 
6((log A'^)/A'^)), the maximum pointwise distortion goes to zero 
as D = 0({\ogNf/N) or D = 0(i?„ei2"''^^^). This is 
shown to be possible with only nearest-neighbor communication, 
distributed coding, and appropriate interpolation algorithms. For 
a fixed, nonzero target distortion, the number of sensors and the 
network rate needed is always finite. 

Keywords: bandlimited fields; nonuniform sampling; dithered 
scalar quantization; oversampled analog-to-digital conversion; 
distributed source coding; sensor networks; scaling law; 



I. Introduction 

Motivation: High resolution remote sensing of physical 
phenomena is a task of considerable importance in applica- 
tions such as environment/weather monitoring, ecology, and 
precision agriculture. Consider the scenario where a large 
distributed network of low-precision, low-power sensors is 
deployed over a region of interest to collect and return 
measurements to a central data collection and processing unit 
(CPUfl over some time duration. The goal is to efficiently 
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acquire, process, and transport data to faithfully reconstruct 
the physical field under appropriate measures of fidelity. 

Physical fields such as temperature, pressure, magnetization, 
and vibration are fundamentally analog in nature. Their spatio- 
temporal distributions obey fundamental physical laws (a 
temperature field must satisfy Laplace's equation, the velocity 
field in fluid flows must satisfy Navier-Stokes equation, etc.) 
that induce strong dependencies across space and time. Many 
physical fields are approximately bandlimited in space and 
time (the physical propagation laws often providing a natural 
filtering effect that attenuates high frequencies) and can be 
reconstructed in a stable manner from samples taken slightly 
above the Nyquist-rate on a uniform lattice. However, in 
practice, the samples of the field are quantized due to the finite 
precision of ADCs, leading to unavoidable field reconstruction 
errors. When bandlimited fields are uniformly sampled at 
the critical Nyquist-rate (in space and time), the worst-case 
pointwise reconstruction error D{R) decays exponentially 
with the bitrate R (measured in number of bits per Nyquist- 
interval per snapshot@of the ADCs [l]-[5], that is, D{R) is of 
the order of 0{2~^). Error can be reduced by increasing the 
quantization resolution of the sensors. Is it possible to sacrifice 
ADC-precision (lower amplitude resolution) for denser spatial 
sampling rate (higher spatial resolution) while maintaining the 
same reconstruction quality and the same number of informa- 
tion bits per Nyquist-interval per snapshot? In the sequel we 
show that it is indeed possible to achieve a flexible array of 
tradeoffs between the (amplitude) resolution and the density 
(spatial resolution) of sensors leading to a "conservation of 
bits" principle. 

The classical problem of sampling of continuous signals is 
a mature topic in signal processing that has accumulated a 
rich knowledge-base over the past several decades [1], [2], 
[6]. However, sensor networks impose challenging constraints 
on the classical sampling paradigm in terms of low device pre- 
cision and power and prohibitive communication costs. This 
biases solutions toward highly distributed and/or localized col- 
laborative processing environments characterized by nearest- 
neighbor communication. This also calls for distributed com- 
pression algorithms to be integrated with sampling. We use 
the term "distributed sampling" to capture these effects. This 
paper is accordingly driven by the goal of addressing the 
sampling problem within the context of sensor networks 
and their associated constraints in terms of both locaUzed 
processing and communication. At an information processing 

^Sensors are assumed to periodically sample the field in synchronism at 
discrete time instants at their locations. A field snapshot refers to the entire 
continuous-space field at one sampling time instant. 
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level, there are two broad functional tasks in data-gathering 
sensor networks: (i) data-acquisition or sampling, processing, 
and coding of the sensor field measurements and (ii) data 
transport, where the acquired data is disseminated across the 
network to a CPU. Both tasks are addressed in this work. 
The data-acquisition task is addressed through a distributed 
sampling framework that is well-suited for sampling sensor 
fields with minimal or no inter-sensor communication. The 
data-transport task is addressed through distributed coding 
and nearest-neighbor communication that moves the sensor 
data to the CPU. For a related information-theoretic study of 
multiuser source coding with distributed vector quantization, 
see [7], [8]. For work with a different flavor on quantization 
and oversampling, see [9]. 

Contributions: (i) "Bit-conservation " principle: A key con- 
tribution of this work is that the bit-budget per Nyquist- 
interval (the rate) can be distributed along the amplitude axis 
(sensor-precision) and space (sensor density) in an almost 
arbitrary discrete-valued manner, while retaining (the best- 
known) exponential distortion-rate characteristics (Section lVTl i. 
This provides the network-architect with a guiding design 
principle for spatially adaptive sampling in terms of selecting 
sensor precision, density, and deployment patterns to meet the 
desired reconstruction quality. We can use higher precision 
ADCs when the sampling density is constrained to be light by 
the underlying topography of sensor-deployment, e.g., rugged 
terrain or occluding obstacles, and use lower precision sensors 
when the sampling density can be high. This also clarifies 
the need for diversity in sensing capabilities for easing the 
adaptation to changing environments. 

(ii) Information scaling law: The pioneering work of Gupta 
and Kumar [10] that has precipitated interest in scaling laws 
for ad hoc networks, applies to scenarios where nodes produce 
independent data, no matter what the scale of the network. 
This does not entirely apply to the sensor network context, 
where the "information density" (related to the sensed field) 
remains fixed regardless of the "network density" (related to 
the network size). The inter-sensor data correlation induced by 
the underlying physics of the phenomenon being sensed can 
be exploited to reduce the net information to be disseminated 
in the network. How does network information grow with in- 
creasing demands on quality? What are the network resources 
needed to sustain high-quality large-scale field acquisition? 
How should the underlying spatio-temporal correlation be 
exploited? For reconstructing a discrete-time sequence of 
spatially bandlimited fields over a fixed bounded region at 
a CPU, we show that the following scaling behavior can be 
realized with one-bit sensors, distributed coding, neighbor-to- 
neighbor local communication, and suitable spatial interpola- 
tion (Section IVII-B.2b : With N one-bit sensors per Nyquist- 
interval, 0(logA^) Nyquist-intervals, and total network bi- 
trate i?„et = 0{{logNf) (per-sensor bitrate e((logiV)/A^)), 
the maximum pointwise distortion goes to zero as D = 
0{ilogN)yN) orD = 0{Rnet2-^'^^). 

(iii) Stochastic fields: An important technical contribution 
of this work is the generalization of the nonuniform sampling 
results for amplitude limited, bandlimited, square-integrable 
deterministic fields due to Cvetkovic and Daubechies [11] to 



amplitude limited, bandlimited, wide-sense stationary (WSS) 
spatial stochastic processes (Sections |III1 IIV-BI IV-BI and 
IVI-Bl i. This generalization is effected by leveraging certain 
results for the Zakai class of bandUmited fields due to Zakai 
[12] and Cambanis and Masry [13]. The maximum pointwise 
error versus rate results are established in the strong almost- 
sure sense (hence also in the expected p-th power sense, 
p G (0,cx))). We build upon the work by Cvetkovic and 
Daubechies along three directions. First, we generalize their 
results to arbitrary precision ADCs. Secondly, we extend their 
framework to the stochastic setting. Thirdly, we leverage these 
results from the classical centralized setup to the distributed 
setting accounting for information transport costs making them 
relevant to the sensor network context. 

Key ideas: A key concept underlying the ability to achieve 
a flexible tradeoff between amplitude and spatial resolution 
is dithered sampling due to Cvetkovic and Daubechies [11]. 
For each field snapshot, sensors add (or compare) the value of 
a pre-designed dither function at their respective locations to 
their observations and note only the sign of the sum. By design 
of the dither function, the sum of the bandlimited field and the 
dither function is ensured to have exactly one zero-crossing 
in every Nyquist-interval. This induces a strong dependency 
in the binary observations of all N = 2^ sensors inside 
each Nyquist-interval. Their joint entropy-rate is ensured to 
be not more than R bits per snapshot. This data can be 
compressed using a distributed source code and moved to 
the CPU using nearest-neighbor communication. The total 
rate for this is not more than R bits per Nyquist-interval 
per snapshot. The spatial resolution of the reconstructed 
zero-crossings, which is proportional to inter-sensor separation 
= 0(2"''^), can be translated to amplitude resolution of 
the reconstructed field at these locations through the local 
smoothness properties of the field and the dither (mean- 
value theorem). Using results in nonharmonic Fourier analysis 
due to Cvetkovic and Daubechies [11], per-sample amplitude 
resolution can be converted, as will be shown, to global field 
approximation accuracy of the same order using interpolation 
techniques. The maximum reconstruction error using 2^ one- 
bit sensors per Nyquist-interval will then be of the order 
of 0{2^^). This is equivalent to the reconstruction accuracy 
of one i?-bit sensor per Nyquist-interval. A similar approach 
using dithered feveZ-crossings instead of zero-crossings allows 
one to increase precision and decrease density maintaining 
order-optimal accuracy. 

Caveats: This work has a sampling-theoretic focus. Al- 
though the motivation is two-dimensional sensor networks, 
for clarity, the theoretical development focuses on a single 
time-snapshot and spatial dimension equal to one. The mul- 
tidimensional case is discussed in the last section. We also 
acknowledge that there are several important problems that 
must be overcome to make the results of this work practically 
viable. This includes sensing-noise, sensor-deployment issues, 
sensor location errors or uncertainties, and sensor synchroniza- 
tion and scheduling issues. We defer the exploration of such 
issues to future research. 

Organization: Field modeling assumptions, distortion crite- 
ria, and assumptions on data-acquisition and communication, 
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are discussed in Section HI] Section |lll] discusses the key 
technical issues and resuhs in going from the deterministic 
to the stochastic setup. Throughout, the deterministic and the 
stochastic scenarios are discussed side-by-side to emphasize 
similarities and differences. Classical deterministic Nyquist 
sampling and its stochastic extension is discussed in Sec- 
tion HV] Section [V] discusses single-bit dithered oversampling, 
first by summarizing the deterministic results by Cvetkovic 
and Daubechies [11] and then extending it to the stochastic 
case. A "bit-conservation" principle and its implications are 
exposed in Section |VT] by showing how a flexible trade-off 
between sensor-precision and sensor density can be achieved. 
Distributed processing and communication issues and costs 
germane to sensor networks are discussed in Section IVIII 
This includes distributed field-acquisition and coding (Sec- 
tion IVII-Al i. information transport and associated achievable 
information scaling law in terms of the bitrate, distortion, and 
sensor density for field reconstruction in a compact region of 
interest (Section IVII-B.2l i. and extensions to two and higher 
spatial dimensions (Section IVII-Q . The main contributions 
are summarized in Section IVIIII Proofs of technical results 
are relegated to the appendices to maintain a smooth flow of 
ideas. 

Notation: Real numbers, the integers, and the natural num- 
bers are respectively denoted by M, Z, and N. The term 
'field' is used to emphasize the dependence of a quantity 
on spatial coordinates, e.g., electromagnetic field, temperature 
field, pressure field, etc. All random variables and processes 
are defined with respect to a common probability space 
(iljJF, P) and E[-] denotes the mathematical expectation oper- 
ator. Random quantities are denoted by capital letters (e.g., X) 
and specific realizations of random quantities by small letters 
(e.g., x). Modifications of f{t) and X{t), e.g., f',X'J,X, 
etc., respectively denote deterministic and stochastic fields and 
t,T,ti,Ti, etc., are used for spatial (not temporal) variables. 
The notation X without adornments denotes the entire pro- 
cess {X(t)}t^M- denotes the space of p-tii power 
ip G (0, oo)), Lebesgue-integrable fields on M."^ where d = 
dimension. BL{S) denotes the space of deterministic, square- 
integrable (finite-energy), continuous, bandlimited field^ on 
K.'', with a compact spectral support set S. The phrase 'almost- 
surely' is synonymous with the phrase 'with probability one'. 
The term bitrate is used to denote bits per spatial Nyquist- 
interval/area/volume for each field snapshot. We follow Lan- 
dau's asymptotic notation: f{x) — 0{g{x)) as x ^ a 
\imsnp^^^\f{x)/g{x)\ < oo; f{x) e(g(a;)) ^ f{x) = 
0{g{x)) and g{x) — 0{f{x)). The value of a is either oo or 
and will always be clear from the context. 

IL Modeling Assumptions and Fidelity Criteria 

We consider a discrete-time sequence of continuous-space 
spatially bandlimited fields with a limited amplitude range. 
Both deterministic and stochastic field models, described be- 
low in Section Hl-AI are studied. These modeling assumptions 

3 In £2(iRd) 3 bandlimited field can be discontinuous on a measure-zero set 
that may even be uncountably infinite. The continuity condition is explicitly 
included due to our interest in sample-inteipolated reconstructions. 



hold for each discrete-time field snapshot with spatial dimen- 
sion d equal to one. An extension to higher spatial dimensions 
is discussed in Section IVIII When dependencies across time 
snapshots are arbitrary, our error versus rate results apply only 
to each snapshot and not to other intermediate time instants. If 
the field is temporally bandlimited, snapshots are assumed to 
be taken at the temporal Nyquist-rate. Then our results apply to 
all spatio-temporal points. Knowledge of the field model and 
associated parameters such as bandwidth and dynamic range 
are assumed to be available during the design-phase prior to 
sensor deployment. Sensors are assumed to be deployed in a 
uniform rectangular grid and the sensor locations are assumed 
to be available to the fusion center. Sensors are assumed to 
operate in a time-synchronous manner Although joint source- 
channel issues and power-distortion considerations are not 
considered in this work, the data-transport costs in terms of 
bitrate are studied in Section IVIII The finite-rate encoded 
sensor observations together with their sensor identification 
labels are made available to a fusion center The objective 
of the fusion center is to estimate the field value at each 
continuous-space location for each discrete-time snapshot. The 
estimation quality is measured by the maximum absolute error 
over all space and time. Bounds on these maximum errors over 
all space and time also imply bounds on related space and time 
averaged distortion criteria described in Section III-BI 

A. Field-models 

The following modeling assumptions apply to each temporal 
snapshot of the field that is acquired by the sensor network. 
For clarity, the theoretical development focuses on a single 
time-snapshot and spatial dimension equal to one. The multi- 
dimensional and temporal aspects are discussed in Section lVTIl 

1) Deterministic case: We assume that f{t), < e K, is 
an amplitude limited field, with amplitude limit A, belonging 
to BL{[-W,W]). Here, both A and W are some finite, 
strictly positive, real numbers. Distributed sampling of de- 
terministic, square-integrable, non-bandlimited fields, with an 
exponentially decaying spectrum, can also be studied using 
the techniques discussed here (see [14] for details), but will 
not be entered into. It should be noted that a limited amplitude 
constraint is less restrictive than a limited energy constraint for 
bandlimited fields. A bandlimited field with Umited energy is 
necessarily amplitude limited but the converse need not be 
true, e.g., Vt G R, f{t) = sin(t). The modeling assumptions 
admit fields with an arbitrarily high but finite energy. 

A property of bandlimited fields that plays an important 
role in the derivation of field-interpolation error bounds from 
nonuniform samples [11], and used in our work, is the 
uniformly bounded slope property. Specifically, according 
to Bernstein's inequality [15], [16, p. 144], for fields in 
BL{[-W,W]) with amplitude limit A, 

\nt)\<Wsnp\f{t)\<WA, VteM. (2.1) 

Through a suitable renormalization of the amplitude and 
spatial axes, it can be assumed, without loss of generality, that 
the amplitude ranges of / and /' are respectively contained in 
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[— 1, 1] and [— 7r,7r] and the spectral support of / is contained 
in [— tTjTt]. This corresponds to W = n and A = 1. 

2) Stochastic case: We assume that X{t),t E M, is an 
amplitude limited stochastic process (field), with amplitude 
limit A, which is WSS with an autocorrelation function 
Rx{t),t e M, belonging to BL{[-W,W]). Note that the 
process X is mean-square continuous since Rx is continuous0 
Although somewhat restrictive, the class of processes which 
are stationary, bandlimited, and amplitude limited, is quite 
rich. Some examples are presented in Appendix |l] 

B. Distortion criteria 

The performance of different sampling and reconstruction 
schemes will be compared by deriving a rate-dependent upper 
bound for the maximum (over space) field reconstruction error, 
that is, the £°°-norm of the error 

1) Deterministic case: Let denote the reconstruction 
of a single field snapshot /(t) at the spatial location t G R, 
according to some sampling, quantization, and reconstruc- 
tion scheme which uses R bits per meter per snapshot. Let 

= (/('■) ~ fR{t)) denote the reconstruction error at t. 
For different sampling and reconstruction schemes, we will 
derive upper bounds 'r]{R), on the maximum (over space t) 
field reconstruction error, D{R) := ||e||oo- These bounds 
then automatically hold for the so-called locally-average^ 
quasi-norrr@ of the reconstruction error e defined by 

\\eUt,T) := U / \e{sWds\ , (0,oo), 

J\s-t\<T/2 J 

because for all < e M and all T G {0,oo), ||e||,(i,T) < ||e||oo. 

2) Stochastic case: Let Xr and E ~ {X — Xr) respec- 
tively denote the reconstruction and error fields according 
to some sampling, quantization, and reconstruction scheme 
which uses R bits per meter per snapshot. It should be 
noted that E may not be WSS because the reconstruction 
scheme may be nonlinear (due to quantization) and spa- 
tially varying (due to sampling). For different sampling and 
reconstruction schemes, we will derive upper bounds rj{R) 
such that ||i?||oo < ^(-R) almost surely and consequently 
also ||(E|£'|P)i/P||oo < ri{R) for any p e (0,oo). As in 
the deterministic case, these bounds will also apply to the 
locally-averaged error (| |£'| T)) both in the almost-sure 
and expected p-th power senses. 

III. From Deterministic to Stochastic Fields 

To expose the intuition underlying the main results of this 
work and for expositional clarity, in the sequel, we first discuss 
results for the deterministic setting, and then immediately 
extend this to the stochastic setting. Moving from the deter- 
ministic to the stochastic setting would have been straight- 
forward if almost all sample paths of X inherited the same 

''Continuity at t = suffices since it implies continuity \/t [17]. 

^This is a detenninistic spatial average and not a stochastic average with 
respect to any probability distribution. 

^Note that this is a norm only for g £ [1, oo). We do not require this to 
be a norm so we allow q € (0, oo). 



properties assumed for deterministic fields. The deterministic 
nonuniform sampling and interpolation results for amplitude 
and bandlimited fields by Cvetkovic and Daubechies [11], that 
we build upon, are stated for square-integrable fields that also 
satisfy a crucial uniformly bounded slope property (Bernstein's 
inequality ( 12.11 ). Moreover, the nonuniform sampling locations 
in these deterministic results, are intricately coupled to field 
values, because they are derived from certain level-crossings 
of the field. It is unclear if these conditions hold in the 
stochastic setting. For instance, while every sample path of 
X is amplitude limited, it need not be bandlimited in the 
conventional sense. In fact, the sample paths need not even be 
square-integrableQ which is a common assumption in results 
pertaining to the approximation of deterministic fields from 
a discrete set of samples. The extension of sampling results 
from the conventional square-integrable bandlimited fields to 
the stochastic case is one of the key contributions of our work. 

It turns out that almost all sample paths of a mean-square 
continuous, WSS stochastic process, with an autocorrelation 
function in BL{[—W,W]), are bandlimited in the sense of 
Zakai [12], [13], [18]. This is an extension of the conventional 
notion of bandlimited fields. The Zakai class of bandlim- 
ited fields (defined below) includes the conventional square- 
integrable bandlimited fields but also includes fields such 
as nonzero constants and pure tones (sinusoids) which are 
not square-integrable. Therefore, our approach for extending 
sampling results from the deterministic to the stochastic setting 
is to first establish deterministic sampling results for the 
Zakai class of bandlimited fields, which includes non square- 
integrable fields, via Lemma [TTl The results for the stochastic 
setting follow immediately, in a strong almost-sure sense, via 
Fact 13.11 below. The results then automatically hold in an 
expected p-th power sense as well. 

Definition 3.1: (Zakai class of bandlimited fields [12], [13], 
[18] ZBL{W,S)) A field /(t), t e R, is said to be 
bandlimited in the sense of Zakai, with bandwidth W and 
a margin parameter S e (0, oo), if \ f{t)\'^j^dt < oo 
and 

/oo 
f{T)h{t - T)dT, (3.1) 
-oc 

where h(t) = sm{{W + 5/2)t) sm.{5t/2) fox t ^ and 
h{Q) := {2n)-^{2W + 5). ZBL{W,d) is the set of all fields 
which are bandlimited in the sense of Zakai with bandwidth 
W and margin parameter 5. 

To provide some insight into the Zakai class of bandlimited 
fields, we summarize some of its salient properties in this 
paragraph. These properties are not needed in the subsequent 
development. The function h appearing in the definition of 
ZBL{W, S) has a bounded amplitude and is both absolutely 
and square integrable because it decays sufficiently fast (as 
It has a Fourier transform given by the trapezoidal- 
shaped function shown in Figure [T] Thus h e BL{[—W — 
5,W + 5]). The set of all Borel-measurable complex-valued 
functions f{t) on the real line satisfying \ f{t)\^jj^dt < 

^If X{t) is a strictly stationary ergodic process with < Rx(0), then 
almost-surely _R(0) = limT^cx)(2T)~i f[_T t] ^^(*)'lt- Thus almost all 
sample paths ai'e not squai'e-integrable. 
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Fig. 1. Fourier transform of kernel h{t). 

oo forms a Hilbert space with inner product < /, g >= 
I^oo -^(^^9* (^^iTt^'^^ where * denotes complex conjugate. 
This Hilbert space includes £^(M), bounded functions, and 
functions for which ^ J^j, |/(t)pdi is bounded in T, e.g., 
f{t) = sin(i), t e M [12]. ZBL{W,S) then forms a 
reproducing-kernel subspace of this Hilbert space with the 
Toeplitz reproducing kernel h. Every function in ZBL{W, S) 
is equal almost everywhere (with respect to the Lebesgue 
measure) to a continuous function (given by the right side of 
( 13. Il l) and, as in the deterministic case (see footnote O, only 
these continuous representatives will be considered. For all 
S,W, BL{[-W,W]) C ZBL{W,S), that is, conventionally 
(square-integrable) bandlimited fields are also Zakai-sense 
bandlimited. It is possible to use other kernels for h, e.g., 
kernels whose Fourier transforms increase and decrease super- 
hnearly over [— — (5, — VF] and [W,W + 6] respectively. 
Extending prior work due to Zakai [12], in [13] Cambanis 
and Masry showed that for all W,S, a field g G ZBL{W,S) 
if and only if, for all t e R, g{t) = .g(0) + tf{t), for some 
/ e BL{[-W, W]). Thus, ZBL{WJ) is in fact independent 
of the value of the margin parameter 6 which can be taken to 
be any arbitrarily small but strictly positive real number and 
mainly serves to simplify convergence issues. 

The following uniformly bounded slope property for am- 
plitude limited fields in ZBL{W, 6), proved in Appendix ITU 
is crucial for establishing the asymptotic exponential decay of 
approximation error with bitrate for the stochastic case. This 
is discussed in Sections |V] and |VT] in the context of sampling 
using low-precision ADCs. The bound provided is sufficient 
for our purpose although it could potentially be strengthened 
to something more like Bernstein's inequality (12. 11 1. 

Proposition 3.1: (Amplitude limited fields in ZBL{W, 6) 
have bounded slope) Let < M^, (5, A < oo, / G ZBL{W, S) 
continuous, \f{t)\ < A for all t G R, and h be as in Defini- 
tion O Then / is differentiable and \f{t)\ < 2AW^ < oo 
for all t G E. 

The following key result, proved in AppendixHIIl shows that 
general uniform/nonuniform sampling and interpolation results 
which hold for amplitude limited fields in BL{[~W,W]), 
will also continue to hold for amplitude limited fields in 
ZBL(W,6), for all W,d. 

Lemma 3.1: (Lifting sampling and interpolation results 
from BL{[-W,W]) to ZBL{W,5)) Let < M^, J, A < oo 
and h G BL{[~W — 5,W + 5]) be the Toeplitz reproducing 
kernel in Definition 13. II Let T a set of sampling 

locations, and a set of interpolation kernels be such 

that (i) C := sup^^jj [X](ez IV^K^ ~ ^01] < °° ^^'^ (ii) for 



(i,r) G 

L 

h{t - t) = lim V h{ti - T)in{t - ti) 

L > oC' ^ — ^ 

(the existence of such T and is ensured by Fact l5.1l l. 

Then for any / G ZBL{W, S) with amplitude limit A, 

L 

fit) = lim V f{tl)^Pl{t-tl), 

L— >oo ^ — ^ 

l = -L 

where the above series converges absolutely for each t and 
uniformly on all compact subsets of M. 

Sampling and interpolation results which hold for fields in 
ZBL{W, 6), also hold true for the sample paths of a WSS 
process with an autocorrelation function in BL{[~W, W]), in 
the almost-sure sense, due to the following fact. 

Fact 3.1: (Sample paths of mean-square continuous ban- 
dlimited WSS processes [12, Sec. 5], [13, Sec. 3]) Let 
< VF, (5 < oo. If X is a WSS stochastic process with an 
autocorrelation function Rx € VF, M^]), then almost- 

surely (with probability one), its sample paths belong to 
ZBL{W,6). 

Thus, if there is an R bits per meter per snapshot sam- 
pling, quantization, and reconstruction scheme such that ||/ — 
/fliloo < viR) for all / G ZBL{W,S) then for any WSS X 
with Rx e BL{[-W,W]), almost-surely, \ \X{t)-XR\\^< 
rj{R). Thus an upper bound on the maximum error that holds 
for deterministic fields which are Zakai-sense bandlimited, 
also holds for bandlimited WSS stochastic fields in a strong 
almost-sure sense. 

As in the deterministic field model, through a suitable 
renormalization of the amplitude and spatial axes, it can be 
assumed, without loss of generality, that the amplitude ranges 
of / and /' are respectively contained in [—1, 1] and [— 7r,7r] 
and the spectral support of / is "essentially" contained in 
[— 7r,7r]. This corresponds to = tt with 5 arbitrarily small 
but fixed, and A = 1/{2W'^). 

IV. Nyquist-sampling (improving quality through 
sensor-precision) 

A. Deterministic case 

Any field / G BL{[~ti,tt\) can be reconstructed from the 
samples according to the well known interpolation 

formula [1] 

L 

f{t)= lim V /(/)sinc(t-0, MteM., (4.1) 

l = -L 

where sinc(t) (l/7rt) sin(7rt) for t ^ Q and sinc(O) := 1. 
The Nyquist-sampling period for / is Txq ~ 1- However, 
in practice the reconstruction ( 14.11 ) is not stable to bounded 
perturbations in the sample values due to the poor decay 
properties of the sine interpolation kernel, that is, the series 
in ( 14.11 ) is not absolutely-convergent. All practical ADCs have 
finite precision and the sampling process is invariably subject 
to perturbations. The instability in the reconstruction implies 
that the quantization noise/error can potentially build up and 
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lead to unbounded reconstruction errors in parts of the fieldH 
However, the instability can be overcome by taking samples 
slightly above the Nyquist-rate; 

Fact 4.1: (Stable interpolation of bandlimited fields from 
samples at uniformly spaced locations [11]) For each 
A > 1/Tmq = 1, there exists an absolutely-integrable 
kernel (t)x{t) belonging to i?L([— ttA, ttA]) such that C :~ 
suPtgM(E;GZ l9^A(i - (V^))l) < oo and for all fields / G 
BlC[--kX + 5,t:\-S\) with < 5 < ttA, 

f{t) = ^lirn^ /(VA)0A - I) , Vi e R, (4.2) 

where the above series converges absolutely for each t (and 
in C^) and uniformly on all compact subsets of M. 

It is due to the finiteness of C that the reconstruction series 
(14.21 ) is absolutely and uniformly convergent. An example of 
an interpolation kernel <f)\ mentioned in Fact l4.1l is the kernel h 
in Definition 13.11 with W = tt and (5 = 7r(A— 1). In fact, there 
exist interpolation kernels (px that decay faster than cit^'- for 
ain e N and some constants {c;}igN- Technically, T = {ti 
l/X}i^z is said to form a set of stable sampling points for fields 
in BL{[—tt,7t]) [2]. This is because bounded perturbations in 
the sample values at these locations never lead to unbounded 
reconstruction errors at any point unlike in ( 14. It where the 
sampling interval is exactly equal to a Nyquist-interval. 

A "near-Nyquist" stable sampling rate A > 1 but close to 
one will be held fixed for the rest of this paper and the term 
oversampling will be used to refer to uniform sampling at a 
spatial sampling rate which is strictly greater than A. The term 
"Nyquist-interval" shall also be used, loosely, to refer to any 
stable sampling interval of the form [l/X, {l+l)/X),l E Z. The 
structure of Nyquist-sampling (for sampling rate A) is depicted 
in Figure 12] To study the effect of finite precision ADCs on the 
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Fig. 2. Classical Nyquist-sampling witli 3-bit ADCs placed at 
Nyquist-locations The entire budget of k — 3 bits is ex- 

hausted at a single sample point in any Nyquist-interval. The spatially 
maximum interpolated reconstruction error decays exponentially in the 
bits per Nyquist-interval R = kX with exponent (1/A). 

reconstruction of unit amplitude limited fields in BL{[—tt, tt]), 
let Qk denote the /s-bit uniform scalar quantization operation 

'^However, if tlie perturbations are only due to scalar quantization where 
is a reproduction point, then the interpolation enor will be bounded for fields 
in BL([-7r,7r]). 



on [-1, 1] [19] so that \z - Qk{z)\ < 2'^ for all z e [-1, 1]. 
The bitrate in bits per spatial Nyquist-interval for each field 
snapshot used to quantize the field is given by i? = fcA. Let the 
bitrate-i? quantized Nyquist-reconstruction of unit amplitude 
limited / e BL{\—'k^tt\) be defined as: 



lim V QkUil/Xm 



l=-L 



Using ( I4.2l i in Fact 14.11 and the triangle-inequality, it immedi- 
ately follows that the spatially maximum reconstruction error 
can be bounded as 



D{R) ■■=\\f 



< C-2 



(4.3) 



Thus D{R) = 0(2--"/^) as i? ^ oo and R{D) = 
0(log2(l/£')) as D ^ 0. This shows that maximum error can 
be made to decay to zero at least exponentially fast, with an 
exponent A^^, as the bits per spatial Nyquist-interval goes to 
infinity. The constant C depends on the choice of interpolation 
kernels. We now argue, informally, that the rate of decay of 
the maximum error cannot be much better than exponential in 
the bitrate. Every sequence {zi}i^z, of values for the samples 
at the Nyquist-locations, essentially determines an amplitude 
limited / in i?L([-7rA, ttA]) via i4^ . with /(//A) w z; for 
all I G Z. Let {zi}i^z be a sequence whose quantization error 
magnitudes \zi — Qk{zi)\, I E Z, are close to 2^'^ and whose 
signs are such that limL_oo Y.f=-Li^i - Qk{zi))4>\{t - l/X) 
is close to C2~^. For such an / we have D{R) w C ■ 2~~^. 
Thus informally. 



sup 11/ 



That is, for Nyquist-sampling with uniform scalar quantization 
and a given choice of interpolation kernels, the maximum 
reconstruction error for the worst amplitude limited field in 
BL{[—tt,tt]) decays exponentially in the bitrate. A more 
precise statement of this result was formally established in 
[20, Sec. II. B] by computing the Kolmogorov e-entropy for 
the class of bounded square-integrable bandlimited functions. 

B. Stochastic case 

Let X be as in Section III-A.2I (also see normalization 
assumptions in the last paragraph of SectionllllTi. Then in view 
of Fact|4l] Lemma O with ti = IX'^ and ipiit) ^ 4)x{t), 
{l,t) e Z X M, and Fact 13. II almost-surely for all sample paths 
of X, we have Vi £ M, 

Xit) = lim XL{t), XlH) = J2 Xil/X)<Px (t - {) ,(4.4) 



l=-L 



where the above series converges absolutely for each t and 
uniformly on all compact subsets of R. In fact, for each t € R, 
the sequence of partial sums X^ in ( |4.4| i, which are random 
variables, converges to X{t) also in the expected p-th power 
sense, p € (0, oo), due to the following fact. 

Fact 4.2: (From almost-sure convergence to expected p-th 
power convergence) If as L oo, almost-surely jX^ — X| 
0, and \Xl\ < A < oo for all L (then automatically \X\ < A 



6 



almost-surely), then by the bounded convergence theorem for 
random variables [21, p. 17], limi,_>ooE|Xi — X\p for 
all p e (0, oo). 

For each t, let the bitrate-i? quantized Nyquist- 
reconstruction be defined as: 

X^^it) -.^ ^lim J2 (X (VA)) 

where for the same reasons as for ( I4.4I ), almost-surely and in 
the expected p-th power sense, p E (0,oo), the above series 
converges absolutely for each t and uniformly on all compact 
subsets of M. Thus as in ( 14.31 ) we have, almost-surely, 

D{R) ^ \\X - X^^W^ < C2-i«. (4.5) 

This also implies that 

||(E|X-X^«|f)i/P|U < C2"i«, 

for all p £ (0, oo). As discussed in Section III-BI the bound 
C2^~^ in (14. 3b and ( 14.5b also applies to the locally-averaged 
reconstruction errors. Since i? oc fc in the Nyquist-sampling 
framework, the reconstruction quality can be improved only 
by using higher precision ADCs. 

V. One-bit dithered oversampling (improving 

QUALITY THROUGH SENSOR DENSITY) 

In the Nyquist-sampling framework of the last section, 
samples are collected at regular Nyquist-intervals and the "bit- 
budget" for each interval is exhausted at a single sampling 
location. In this framework, the reconstruction accuracy can be 
improved only by improving the precision of the ADCs of the 
sensors. High-precision ADCs are expensive and have higher 
power requirements. Future sensor networks are envisioned to 
be made of cheap, low-power, low-precision devices deployed 
in large numbers over a given geographical area of interest. 
The question which arises naturally in this context is whether 
it is possible to realize high-resolution field reconstruction 
using a dense network of low-precision sensors. Can the 
fixed sensor-precision be adequately compensated through 
high sensor-density? If so, how does the reconstruction quality 
scale with the sensor density? In particular, can the maximum 
reconstruction error over space be made to decay exponentially 
with the bit-budget per Nyquist-interval as in the Nyquist- 
sampling framework? The answers to these questions will 
help characterize the "information density" in bits per meter 
associated with sampling a bandlimited field independent 
of sensor-precision. The focus of this section is on high- 
resolution field reconstruction using one-bit-precision sensors 
in the deterministic and stochastic cases. We first summarize 
the results for the deterministic case due to Cvetkovic, and 
Daubechies from [11] and then develop the stochastic coun- 
terpart. In the next section we study the general case of field 
reconstruction with 6-bit-precision sensors in the deterministic 
and stochastic cases and expose an underlying "conservation 
of bits" principle. 

There is a vast body of literature available on classical one- 
bit oversampled analog-to-digital conversion. This includes (i) 
oversampled sigma-delta modulation used in audio consumer 



equipment for many years [3]-[5], [22], and (ii) oversampled 
randomized dithered averaging [6], [23]-[26] which has con- 
nections with stochastic resonance theory in physics, nonlinear 
dynamical systems, neural sensory information processing 
[27]-[29], and has been used for real-time synthetic-aperture- 
radar imaging [30], [31] (also see and references therein). 
One Umitation of methods like sigma-delta modulation is that 
they are not amenable to distributed implementation because 
sampling is done in a sequential manner where the result of 
quantization at the previous sampling location needs to be 
available at the next location for quantization. A second lim- 
itation is that in all these methods, the bitrate is proportional 
to the oversampling rate. As a result, they have a polyno- 
mial distortion-rate decay characteristic, example, D{R) = 
0(i?^'^™+^)) for m-stage oversampled sigma-delta conversion 
[3], [4], [22] where D{R) is the maximum reconstruction error 
and R is the bit-budget per Nyquist-interval. Hence, there is 
a fundamental exponential performance gap between classical 
methods and Nyquist-sampling (compare with ( 14.3b ). 

A key idea for achieving an exponential distortion-rate 
decay characteristic, as in (14.3b . using fixed-precision ADCs, 
is dithered oversampling. This idea was first proposed by 
Cvetkovic and Daubechies in [11] for the deterministic case 
using single-bit ADCs. The details of this idea and our 
extensions to the stochastic case are described in the following 
subsections. Dithered oversampling with multi-bit ADCs is 
discussed in the next section. 

A. Deterministic case 

Let / be any unit amplitude limited field in _BL([— tt, tt]). 
The key component of dither-based sampling schemes is the 
dither field d{t) that has the following properties; 

1) V/ e Z, 1< 7 \dil/X)\ < oo; 

2) V/ e Z, sign[d{l/X)] ^ -sign[d((Z + 1)/A)]; 

3) d{t) is continuous everywhere and differentiable ev- 
erywhere except possibly in {(VA)};^^ where it is left 
and right differentiable. All differentials are uniformly 
bounded in magnitude by A e (0, oo). 

For example, d{t) = 7COs(A7rt), t G R, with I7I > 1 is a 
valid dither field. Sensors "add" the value of a pre-designed 
smooth dither field d{t) , t e R, at their respective locations 
to the value of their observations f{t) and note only the sign 
of the sum. In terms of hardware, this can be implemented 
using threshold comparators based on operational amplifiers. 
Since / is continuous, the dithered field [/ + d] is continuous. 
The first two properties of d and the unit amplitude limit on 
/ guarantee that [/ + d](//A) and [/ + d]{{l + 1)/A) will 
have opposite signs. By the intermediate value theorem for 
continuous functions [32], it follows that [/ + d] will have a 
zero-crossing (at least one but possibly more) in every open 
Nyquist-interval (1/ X, (l + I)/ X). 

Let N = 2^ (k > 1), one-bit ADCs be placed uniformly 
in every Nyquist-interval to record the sign of the dithered 
field [/ + d], that is, sensors are placed at the locations t^Z 
where := (1/A)2~'^ is the uniform oversampling period. 
Typically r <C 1/A. To avoid clutter, we shall henceforth 
drop the subscript k in t^. Let mi e {0, . . . , {N — 1)} be 
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the smallest index for which [/ + d]{{l/X) + mir) and [/ + 
d]{{l/X) + {mi + 1)t) have opposite signs in {l/X, {I + 1)/A). 
It follows from the intermediate value theorem that for each 
I e Z, f{zi) + d{zi) = at some point z/ G {{l/X) + 
miT,{l/\) + {mi + l)r). To avoid ambiguity, let z; be the 
location of the^rif (from left to right) zero-crossing of [Z + d] 
in the interval [1/ X, {l + l)/ X]. The structure of one-bit dithered 
sampling is depicted in Figure [5] Uniformly amplitude lim- 

Oversampled 1-bit dither 




I /+! 1+2 

AAA 



Fig. 3. One-bit dithered sampling with 8 one-bit sensors separated 

by r = 1/(8 A). The locations of the Grst zero-crossing (from the left) 
in each Nyquist-interval are denoted by zi in the figure. The spatially 
maximum interpolated reconstruction error decays exponentially in k 
with exponent ( 1/ A) as in the Nyquist sampling framework. 

ited and bandlimited fields have uniformly amplitude limited 
derivatives. Specifically, according to Bernstein's inequality 
(12.1b in Section III-AI for unit amplitude limited fields in 
BL([-7r,7r]), Vi £ M, < tt. The third condition on 

the dither field and ( 12.1b together ensure that the value of 
the derivative of [/ + d] is uniformly bounded by (tt + A). 
Applying Lagrange's mean value theorem [32] to the mid- 
point of the "uncertainty zone" ti := {{l/X) + {mi + (1/2))t) 
and zi, we get 

|[/ + d]{zi) - [/ + d] {ti)\ < (tt + A) \zi -ti\, that is, 

lfM-i-mm<-{^y-{^^y^ 

Thus, uniform oversampling of the dithered field using one- 
bit ADCs gives samples of / having linear precision in 
T (exponential in k) at the nonuniformly spaced locations 
^ = {ti}i£i. which depend on both the field and the 
dither Loosely speaking, the dither field provides a way of 
translating spatial resolution/uncertainty of the zero-crossing 
locations (but amplitude certainty, equal to zero) to amplitude 
resolution/uncertainty of the field at these locations (but spatial 
certainty, equal to the mid-point of the uncertainty zone). 

Unlike Nyquist-sampling where the bit-budget for each 
Nyquist-interval is exhausted at a single sampling point, the 
dither-based approach "spreads" the bit-budget over many 
single-bit ADCs in a Nyquist-interval. There are N = 2^ one- 
bit sensors uniformly distributed over an interval of length 
(1/A). It takes k bits or a biti-ate of i? = kX = 0{\ogN) bits 
per Nyquist-interval per field snapshot to index the location 
of the first zero-crossing. Deferring a full discussion of 
distributed processing and communication issues and costs to 
Section rvm as a conceptual aid one may imagine that when a 



sensor observes a sign change relative to its neighbors, it can 
wirelessly broadcast its identity to the CPU using log N bits 
with all other sensors remaining silent. For each I e Z, the 
sample errors decay with rate R as follows 

\f{ti)-{-d{ti))\ < (^^Ay-ii^, (5.1) 

This per-sample amplitude resolution, which is proportional 
to inter-sensor separation, can be converted to global field 
amplitude resolution of the same order using the following 
result on field interpolation from nonuniformly spaced samples 
(adapted from [11, Theorem 3.2]) due to Cvetkovic and 
Daubechies. 

Fact 5.1: (Stable interpolation of bandlimited fields from 
samples at nonuniformly spaced locations [11, Theorem 3.2]) 
Let A > 1 and T = be a set of sampling loca- 

tions such that K := — U\ > and R := 

sup;g2 l^i ~ ( VA) I < oo. Then there exist absolutely-integrable 
interpolation kernels ipi belonging to BL{[—ttX, ttA]) such that 
C" := supjgR [X^igz ~ ^01] < and for all fields 
/ e BL{[-TrX + S,7rX-S]) with < 6 < ttX, 

L 

/(t)= lim V f{ti)Mt-ti), 

L ^oo ^ — ^ 

l=-L 

where the above series converges absolutely for each t (and 
in C^) and uniformly on all compact subsets of R. The 
interpolation kernels ^i depend on the sampling set T but 
there exist interpolation kernels for which C depends only 
on A, K, and R and not on the specific sampling set T. 

In essence, the field can be reconstructed in a stable manner 
as in the Nyquist-sampling framework as long as the sampling 
locations do not get too close to each other (not closer than 
K>0) and do not run away too far from the Nyquist-points 
(not farther than < k < oo). There exist interpolation 
kernels ipi that decay faster than (cj^/i™) for all m where 
the constants {cJ„}mgN depend only on A, k, and R and not 
on T [11]. As discussed in Section rVll-B.2l these properties 
prove useful in the practical context of reconstructing the 
bandlimited field to a desired nonzero distortion from samples 
collected in only a finite spatial region of interest. 

In Remark Isil below, it is shown that T = {ti := {{l/X) + 
{mi + {1/2))t)}i^z satisfies the conditions of Fact 15.11 with 
K, R, and C which do not depend on r. Hence, from ( 15. lb . 
Fact 15.11 and the triangle inequality, it follows that there exist 
interpolation kernels {ipi}iez (which depend on T in general) 
for which the spatially maximum interpolated reconstruction 
error is bounded as follows (also see [11, Corollary 3.3]) 

D{R):=\\f-fr"\\^ < ^'(^)2"*^ (5.2) 
where ^ (-d (^0) V'z - ■ 

The reconstruction error decays exponentially with the bitrate, 
with the same exponent as in the Nyquist-sampling scheme 
( 14.31 ). The reconstruction accuracy can be improved by reduc- 
ing r, that is, by packing more sensors inside each Nyquist- 
interval. However, unlike the Nyquist-sampUng scheme, there 
is no need to use higher precision ADCs. 
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Remark 5.1: We are interested in the behavior of the re- 
construction error as the number of sensors iV = 2'^ = 2^/^ 
increases. Since C depends on k = ii^fj^i l^j ^ U\ 
R := sup;g2 |t; — (but the interpolation kernels ijji^, 

can be designed so that C does not depend on the specific 
nonuniform sampling set), to truly assert that the decay of the 
reconstruction error is exponential in R, we must ensure that 
K and R do not depend on r = {l/\)2~^. Clearly, R < 1/A. 
What about k? A lower bound on k is t since the {ti}i^z 
cannot get closer than t/2 to the endpoints of the Nyquist- 
interval by design. This lower bound depends on t. However, 
since > \d{t)\-\f{t)\ > a zero-crossing 

cannot occur unless the amplitude of d falls below unity. Since 
\d{l/X)\ = 7 > 1, the closest that a zero-crossing can get 
to a Nyquist-point is (7 — 1)/A where A is the maximum 
slope of d. Since the tis are midpoints of inter-sensor intervals, 
K > (2(7—1) /A) — r. Together with the earlier lower bound 
K>T, this means that k> (7— 1)/A which is independent of 
r. For aiding comparison with 6-bit dithered oversampling of 
the next section, we take C to correspond to the conservative 
values K™"" = 1/A and k™"" = min(l/(2A), (7 - 1)/A). 

Remark 5.2: From i5.2i . we would like the maximum slope 
A of the dither field to be as small as possible to get a tighter 
upper bound on the error The average slope of the dither field 
in any Nyquist-interval is (27 A). Since d is smooth on every 
open Nyquist-interval, by the mean-value theorem, there is 
always some point in the open interval where the slope of d 
matches the average slope. Hence, A can be no smaller than 
(27A). We can make A = (27A) by using a piece-wise linear 
dither field. 

Remark 5.3: It is in fact possible to design a dither field 
d for which [/ + d] will have exactly one zero-crossing in 
each Nyquist-interval: consider for example the continuous 
periodic triangular dither field which is piecewise linear in 
each Nyquist-interval and whose slope A is strictly greater 
than TT in magnitude. For such a d, the sum field [/ + d] 
is strictly monotonic in each Nyquist-interval because the 
slope of the sum field never changes sign in any Nyquist 
interval. For expositional simplicity and to aid the discussion 
of communication issues and costs in Section IVIII for the 
remainder of this paper, we will assume that the dither field 
satisfies this property. 

B. Stochastic case 

Let X be as in Section III-A.2I (also see normalization 
assumptions in the last paragraph of Section HIH). It should be 
noted that X is a random process and that its sample paths are 
not bandlimited in the conventional sense. There may even be 
discontinuous sample paths (having probability zero) for which 
[X + d] may not have an actual zero-crossing. However, since 
\X\ < 1 and 7 > 1, [X + d] (l/X) and [X + d] {{l + l)/X) will 
always have opposite signs. Hence, in each Nyquist-interval, 
there always exists a first sensor which will detect a sign- 
change. As in the deterministic case, let Mi be the index (a 
discrete random variable) of the sensor located just left of the 
first sign-change and Ti := [{l/X) + {Mi + (1/2))t]. Note 
that the set T = {Ti}i^z is a countable set of random "zero- 
crossings" (more accurately, first sign-change locations) which 



is closely coupled to the underlying spatial stochastic process 
X. However, each T/ can take values only in the deterministic 
discrete set {{l/X) + (m + 0.5)t : m = 0, . . . ,{N - 1)}. 

From Fact 13.11 and the Remark 15.11 almost-surely the 
random sampling set T satisfies the conditions of Fact 15.11 
Then from Fact 15.11 and Lemma 13.11 with C = C , almost- 
surely for all sample paths of X, \/t e M, 

L 

X{t) = lim y X{Ti)i:i{t-Ti), (5.3) 

l=-L 

where the above series converges absolutely for each t and 
uniformly on all compact subsets of R. In fact, due to Fact 14.21 
for each t E M., the sequence of partial sums to the right of 
( 15.31 ). which are random variables, converges to X{t) also in 
the expected p-th power sense, p G {Q,oo). 

From Fact l3.1l and Proposition l3.1l almost-surely, all sample 
paths of X are differentiable everywhere, and the derivative 
X' has a normalized amplitude which is uniformly bounded 
by TT. Hence, by ( 15.11 ), almost-surely for all paths of X, 

\X{Ti)-{~d{Ti))\ < (^^Ay-iR^ (5.4) 

yi e Z. For each < e M, let the bitrate-i?, one-bit quantized 
reconstruction of X be defined as: 

where for the same reasons as for ( 15.3b . almost-surely and in 
the expected p-th power sense, p e (0, 00), the above series 
converges absolutely for each t and uniformly on all compact 
subsets of M. Thus as in ( 15.21 ). almost-surely, 

= ||X-%*||oo < C"2-i«, (5.6) 
with C" :== C'{n + A)/(2A). This also implies that 

||(E|X-%*|P)i/f|U < C"2-x«, 

for all p e (0,00). Once again, as discussed in Section Hl-BI 
the bound C"2^~^ in ( 15.2b and ( 15.6b also applies to the 
locally-averaged reconstruction errors. It should be noted that 
C" depends only on A, k, and R, which are fixed, and not on the 
specific realization of the set of random sampling locations T 
which satisfy the assumptions of Fact l5.1l However, in general 
the set of interpolation kernels {V'/}iez depends on the random 
sampling set T. 

Discussion: In the stochastic one-bit dithered oversampling 
scenario of this and the following section, the sampling 
locations T and the associated interpolation kernels are tightly 
coupled to the underlying process X because they arise as the 
locations of level-crossings of the process X. Most results 
in the nonuniform sampling literature (see [6], [25], [26] 
and references therein) address the reconstruction of (i) de- 
terministic bandlimited fields from nonuniform deterministic 
sampling locations, (ii) bandlimited stochastic processes from 
nonuniform deterministic sampling locations, (iii) determinis- 
tic bandlimited fields from random sampling locations, and (iv) 
bandlimited stochastic processes from independent random 
sampling locations. Scenarios (iii) and (iv) may arise, for 
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example, due to random sensor-deployment independent of 
the field being sensed. However, the results here are for the 
reconstruction of WSS bandlimited stochastic processes from 
random sampling locations which are strongly dependent on 
the process being sampled. 

Summarizing ( 14.31 ), ( 14.5b , ( 15.21 ), and ( 15.61 ), it is clear that 
for a fixed A, 7 > 1, the C°° norm of the point-wise 
error in the Nyquist-reconstruction or one-bit dithered re- 
construction decays at least exponentially fast in the bitrate 
(bits per spatial Nyquist-interval for each field snapshot) R 
with exponent (1/A). Similar rate-error characteristics hold 
for the deterministic and expected local-average distortion 
criteria of Section [III Hence, if D denotes the maximum 
point error, the "intrinsic information density" (ignoring data- 
transport issues) R{D) in bits per meter grows no faster 
than (with the almost-sure/expected qualifiers in the stochastic 
case) R{D) = 0(log(l/£')) as D decreases to 0. For one- 
bit dithered sampling, the number of sensors per Nyquist- 
interval N, grows no faster than 0{\/D), or equivalently, 
D{N) ~ 0{\/N). The total information in bits per Nyquist- 
interval grows no faster than O(logiV) and the bits per sensor 
decreases at least as fast as 0{{\/N) log A^). 

VI. Tradeoffs in amplitude and spatial 
resolution: a "bit-conservation" principle 

In Nyquist-sampling, fields are sampled at "low", near- 
Nyquist-rates. The entire bit-budget of k bits per Nyquist- 
interval is spent in recording the field amplitude at a single 
high-precision (A: -bit) sensor. On the other hand, the one- 
bit dither-based sampling scheme spends all available bits in 
recording the location of a zero-crossing (a spatial event) by 
using many (A^ = 2*^) poor precision (one-bit) sensors in 
each Nyquist-interval. These sampling schemes represent two 
extreme scenarios. This section explains how fc-bit Nyquist- 
sampling accuracy can be achieved using &-bit ADCs and an 
appropriate dither-based oversampling scheme for any 1 < 
b < k. This leads to a bit-conservation principle - a trade- 
off between the oversampling factor and ADC-precision for 
"similar" asymptotic reconstruction accuracy. To the best of 
our knowledge, this tradeoff has not been discussed before in 
the sampling literature. We discuss this in detail for determin- 
istic fields. The extension to the stochastic setting parallels the 
stochastic extensions in Sections IIV-BI and IV-BI 

A. Deterministic case 

Nyquist-sampling uses only one fc-bit ADC per Nyquist- 
interval of length 1/A. The one-bit dithered sampling scheme 
uses N ~ 2^, one-bit ADCs distributed uniformly over the 
same interval, that is, the ADCs are placed at intervals of 
length T = 1/(A^A). For definiteness, assume that the sensors 
are placed at the beginning of every r-length interval, that 
is, at locations {mTjmgz- Now consider the scenario where 
TVfe := 2'=-''+i 5-bit ADCs (1 < 6 < k) are placed t apart, in 
the leftmost section of the Nyquist interval Notice that, one- 
bit ADCs only detect one level-crossing (the level), 2-bit 
ADCs can detect 3 distinct level-crossings, and in general, b- 
bit ADCs can detect (2'' — 1) distinct level-crossings given by 
{0,±(l/2''-i),...,±(l-(l/2''-i))}. 



Appendix |IV] shows how to design a 5-bit dither field db, 
with maximum slope magnitude A independent of b, so that 
[/ + dt] always crosses some quantization level in every 
interval of the form [Ai,Bi] [l/X, {I + {Nb - l)/iV)/A] C 
[l/X, {I + 1)/A]0 Each interval [Ai,Bi] usually covers only a 
small fraction of the length of a Nyquist-interval and contains 
Nb 6-bit ADCs. The situation is graphically illustrated in 
Figure ID 

Let mi e {0, . . . , {Nb — 2)} be the smallest index for which 
[f+db]{Ai+miT) and [f+db]{Ai + {mi + l)T) are on opposite 
sides of some level qi G {0, ±(l/2''-i), . . . , ±{l~{l/2''-^))} 
and let zi E {Ai + miT,Ai + (m; + 1)t) be the actual point 
of level-crossing, that is, /(z;) + db{zi) = qi. From the mean 
value theorem applied to [/ + db]{t) for the end-points ti := 
Ai + {mi + (l/2))r and zi we obtain 

\[f + db]{zi)-[f + db]{ti)\< sup \[f' + d[]{t)\\zi~ti\, 

t£(Ai,Bi) 

which leads to, 

\f{tl)^{qi-db{ti))\ < 

This shows that the accuracy of the nonuniform samples 
{f {ti)} is linear in t, independent of the precision of the 
ADCs just as in the one-bit dithered samphng scheme. 

It requires only [log2(iVf, — 1)] < {k — b + 1) bits per 
Nyquist-interval to specify the location of the sensor just 
following the location of the first level-crossing of [f + db]{t). 
It takes no more than log2(2'' — 1) < b bits to index the 
level that was crossed first in each Nyquist-interval. Hence, 
the total number of bits required with this sampling method is 
not more than {k-b+l) + b= (fc + 1) bits (or R = A(fc + 1) 
bits per interval) which gives the same distortion-rate decay 
characteristics as the fc-bit Nyquist-sampling and the one- 
bit dither-based sampling schemes. As in Section |V] we 
defer a full discussion of communication issues and costs 
to Section IVIII As a conceptual aid, however, one may 
imagine that the sensor observing a level-crossing relative to 
its neighbor wirelessly broadcasts its index and the index of 
the level that has been crossed to the CPU using {k + 1) 
bits with all other sensors remaining silent. The source of this 
additional bit can be explained as follows: In one-bit dithered 
sampling, the ADCs do not need to explicitly specify which 
level was crossed since there is only one level; only zero- 
crossing locations need to be described. In the fc-bit Nyquist- 
sampling, the locations of the ADCs need not be explicitly 
specified since they are known; however, the sensors need to 
specify the quantization interval in which the field sample lies. 

Since r and k are related via r = 1/(A2''), therefore, r = 
(2/A)2~A^, and the maximum sample error is no more than 

\f{tl)-{qi-db{ti))\ < |^^±^^2-x« (6.1) 

'in fact, it is possible to construct using any dither field d{t) = di{t) 
of the one-bit sampling scheme via Equation jIV. 1 1 in Appendix IIVI Then, 
the maximum slope of the b-bit dither field in any interval {Ai,Bi) is no 
more than A = (4c/7)A, c > (1 + 7r)/4, where A is the maximum slope 
of the one-bit dither in any interval {l/X, {I + 
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Fig. 4. Amplitude resolution versus over sampling rate tradeoffs in conventional and dither-based sampling frameworks, (a) 

Conventional (Nyquist) sampling using 3-bit ADCs placed at Nyquist-locations {l/X}i^z- The entire budget of 3 bits is exhausted at a single 
sample point in any Nyquist-interval. (b) Dither-based sampling scheme using 8, one-bit ADCs distributed uniformly over a Nyquist-interval to 
locate the zero-crossing zi of the dithered held, (c) A flexible tradeoff between these extremes. Four, 2-bit ADCs distributed uniformly over half 
the Nyquist-interval detect a level-crossing zi at level +(1/2). All three schemes have similar exponential error accuracy in the bitrate. 



\fl G Z. Following the analysis for the one-bit case, it can 
be verified that {t;};gz satisfies the conditions of Fact 15.11 
Specifically, fi < (Bi-Ai) = l/(A2(''-i)) < 1/A = k™* and 
> {Ai+i - Bi) > (1 - (l/2(^-i)))(l/A) > 1/(2A) > Ai™- 
(see Remark ISTt . Hence, if the rate-i?, 6-bit reconstruction is 
given by /|j^"(i) Ezez (« " 4 (*/)) {t - U), 

||/-/riloo<C'(^^^2-i^ (6.2) 

where the constant C (tilde is used to distinguish the 
constants of this section from those in Section |V]i does not 
depend on r, /, and the individual {t;}igz but only on 
K, R, and A. If we use the conservative values R = k""''" 
and K = k"" ''" then C = C". Hence, independent of b, the 
distortion-rate asymptotics are the same as in the Nyquist- 
sampling and the one-bit dithered sampling set-ups, that 
is, the distortion is exponentially decaying in rate R with 
the same exponent. This observation is summarized in the 
following principle. 

"Conservation of bits" principle: Let k be the bit-budget in 
terms of the total number of bits available per Nyquist-interval. 
For each 1 < 5 < A: there exists a (dither-based) sampling 
scheme with not more than 2''-''+^, 6-bit sensors per Nyquist- 
interval achieving a spatially maximum reconstruction error D 
of the order of 2^*^. Alternatively, if there are Nb, 6-bit sensors 
in each Nyquist-interval, then D = 0(2-(''+i°S2 ^f)). The 
maximum error decays exponentially in the bit-budget. The 
bit-budget can be apportioned in a flexible manner between 
amplitude resolution in bits 6 and the spatial resolution in bits 
log2 iVf,. It should be noted that Nf, is the sensor density in 
terms of the number of sensors per Nyquist-interval. 

B. Stochastic case 

The procedure for extending the results from the determin- 
istic to the stochastic 6-bit setting is identical to the stochastic 
extension of 



the one-bit deterministic dither scheme. X is sampled by 
adding db having the same properties as in the deterministic 
case. Let Mi be the index of the sensor just prior to the first 
level-change, T; := [//A+ (A/; + (l/2))r] the mid-point of 
the interval in which the first level change is detected, and qi 
be the level crossed. 

From Fact 13.11 and Remark 15.11 almost-surely the random 
sampling set T satisfies the conditions of Fact 15. II In addition, 
from Fact 13.11 and Proposition 13.11 almost-surely, all sample 
paths of X are differentiable everywhere, and the derivative 
X' has a normalized amplitude which is uniformly bounded 
by TT. Hence, almost-surely for all paths of X, the expansion 
( 153b holds, and by dOT i. 

\X{Ti)~{qi-db{Ti))\ < |^^±^j 2-i«, (6.3) 

V/ G Z. For each t G R, let the bitrate-i?, 6-bit quantized 
reconstruction of X be defined as: 

(6.4) 

where for the same reasons as for (15.3b . almost-surely and in 
the expected p-th power sense, p G (0, oo), the above series 
converges absolutely for each t and uniformly on all compact 
subsets of M. Thus as in ( 16.21 ). almost-surely, 

D(R) = \\X - X'j.'-'^W^ < C"2-i^^ (6.5) 

with C" := {C'in + A)/A). This also impUes that 

m\x - x'j,'ry^p\\^ < ^"2-*^, 

for all p G (0,00). Once again, as discussed in Section Hl-BI 
the bound C"2^~^ in ( 16.2b and ( 16.5b also applies to the 
locally-averaged reconstruction errors. It should be noted that 
C" only depends on A,k, and R, which are fixed, and not 
on the specific realization of the set of random sampling 
locations T which satisfy the assumptions of Fact 15.11 
However, the set of interpolation kernels depends, in 

general, on the random sampling set T. 
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C. Discussion 

Nyquist-sampling as a special case: Note that the 
Nyquist-sampling scheme is also subsumed by the proposed 
generalized dithered sampling framework. Indeed, for b = k, 
the described framework suggests using two, fc-bit ADCs at 
locations (j) and (j) + r. However, the dithered field is 
guaranteed to have a level-crossing in [ji j + ''')• Hence, the 
second sensor is redundant and there is no need for the first 
sensor to add the dither-value. 

Number of sensors vs ADC-precision and sensor distribu- 
tion: While maintaining the same asymptotic error decay pro- 
file, for a given reconstruction quality, as the ADC-precision 
b increases, the number of sensors decreases exponentially 
with b. One-bit dithered sampling needs N sensors, 6-bit 
(b > 1) dithered sampling needs Nb ~ N/2^~^ sensors, 
and Nyquist-sampling needs only one sensor per Nyquist- 
interval. The number of sensors can be traded off with the 
precision of sensors. Although the number of sensors has been 
a power two in our analysis, this is only for ease of illustration 
and is not a restrictive assumption. Also, sensors need to 
be placed only in intervals of the form [j, j + (^6 — I)''')- 
This leaves "inactive" regions over which the field is not 
sampled. Although the discussion has, thus far, focused on 
sampling all the Nyquist-intervals in an identical fashion, 
this need not be so. Some Nyquist-intervals can use fewer 
sensors of higher precision {b large) bunched closer together 
in a smaller interval. Others can use more sensors of lower 
precision {b small) spread over a larger interval. The key result 
which enables such a general non-uniform sampling using 
heterogeneous sensors while achieving the same exponential 
rate-distortion decay characteristics is Fact 15.11 This bunched 
irregular sampling ability provides considerable flexibility in 
selecting spatially adaptive sensor precision and deployment 
patterns. For example, in difficult terrain or in the presence of 
occluding obstacles, where sampling density is forced to be 
low, one can use high-precision sensors, and use low-precision 
sensors, in terrain where the sampling density can be high. 

Two other side-benefits of the dither-based oversampling 
scheme worth mentioning (but not quantified here) include 
(i) a certain degree of natural robustness to node failures: if 
every alternate node fails, the effective inter-node separation 
increases to 2r and is equivalent to halving the precision 
of the ADCs and (ii) some measure of data security against 
eavesdropping: the dither fields can be kept covert and there is 
considerable flexibility in their design for a given reconstruc- 
tion quality. 

Table H] summarizes all the salient features of different 
sampling schemes considered in this paper 

VII. Distributed Processing and Communication 
Costs 

Sections IIVI4VII focused on the quantization and inter- 
polation aspects of sampling. In this section we focus on 
distributed processing and communication aspects of dithered 
oversampling for reconstructing a sequence of fields at a 
CPU in a sensor network. We first discuss the problem of 



field acquisition and coding (Section [VII- Al l. We describe two 
approaches for processing and coding raw sensor data, one 
based on local nearest-neighbor communication and the other 
on distributed source coding, and derive per Nyquist-interval 
per snapshot costs. Since spatially bandlimited fields are not 
spatially limited, it is impractical to move data from an infinite 
number of spatial Nyquist-intervals to the CPU. We address 
this issue by considering how the field reconstruction error in a 
compact region of interest decays as the field is sampled over a 
growing neighborhood of sensor deployment (Section [VII-Bb . 
Here we focus on multihop communication to move all the 
processed and coded data from each spatial Nyquist-interval to 
the CPU. We also indicate, however, how a Gaussian multiple 
access uplink may be used for the same purpose. 

To simplify the presentation, we focus on one-bit sensors 
and 1-D deterministic fields in _BL([— tt, tt]) and assume that 
the dither field is such that in each Nyquist-interval (i) there is 
exactly one zero-crossing (see Remark 1531 ) and (ii) the zero- 
crossing direction (the sign change from say, plus to minus, 
moving from left to right) is fixed. In Section IVII-CI we briefly 
indicate how results can be easily extended to two and higher 
dimensional spatially bandlimited fields by reusing the results 
for 1-D fields. 

A. Field acquisition and coding 

Let N, one-bit sensors be placed uniformly at every t = 
meters in every Nyquist-interval [j, -^^^ — t]. Periodically, 
the sensors synchronously take snapshots of the 1-D spatially 
bandlimited random field by comparing the field-value to the 
dither-value at their respective locations. The dither-values are 
assumed to be pre-stored during sensor deployment. Sensor 
clocks are assumed to be synchronized during deployment. 
The temporal sequence of the signs of the field plus dither- 
values constitutes the raw data produced at each sensor 

We consider two approaches for processing and coding 
the raw data produced at each sensor for communication to 
a CPU where the field snapshots are to be reconstructed. 
The first approach involves (a) neighbor-to-neighbor local 
communication within each Nyquist-interval to locate the 
zero-crossings followed by (b) coding of the zero-crossing 
data at each sensor. The second approach does not require 
inter-sensor collaboration and is based on the principles of 
distributed source coding due to Slepian and Wolf [33], [34] 
(see [35] for practical code constructions). Distributed source 
coding exploits the correlation present in the sensor data, e.g., 
there is exactly one zero-crossing in each Nyquist-interval. 
Slepian and Wolf proved that the compression-efficiency of 
non-collaborative coding, in terms of the average rate, is 
equal to that of collaborative coding for discrete-valued data. 
The probability of correctly decoding data which has been 
compressed using a distributed source code tends to one as the 
number of snapshots (coding blocklength) approaches infinity. 

1} Coding using local communication: (a) Locating zero- 
crossings. The sensors at the locations { designated 
as the "leading" nodes for initiating local communication (con- 
fined to Nyquist-intervals) for determining the zero-crossing 
within each spatial Nyquist-interval for each time snapshot. 
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TABLE I 

Comparison of different sampling schemes . 
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Corresponding to each temporal snapshot of the field, the mes- 
sage passed by each node to its neighbor (the right neighbor for 
definiteness), indicates two things: (i) whether or not a zero- 
crossing has already been found by some preceding sensor 
and (ii) if a zero-crossing has not yet been found, the sign of 
the field plus dither at its location. The local communication 
terminates at the last (right-extreme) sensor in the Nyquist- 
interval. In each Nyquist-interval, the first sensor that detects 
a sign change between what its left neighbor reports and its 
own reading records a "1". Other sensors record a "0" for that 
snapshot. Thus, at the fixed inter-sensor communication rate 
of 2 bits per snapshot, the raw one-bit sensor readings can 
be converted to zero-crossing information. Note that the local 
communication can be done in one shot after aggregating the 
data across many snapshots. 

(b) Compression of zero-crossing data. Each sensor encodes 
(compresses) the zero-crossing information - the temporal 
sequence of zeros and ones where one indicates the presence 
of a zero-crossing and a zero the absence. This compression 
can again be done in one of two ways. The first approach, 
not discussed here (but see next subsection), involves us- 
ing distributed source coding by exploiting the correlation 
in the zero-crossing information. However, an alternative to 
distributed source coding involves each sensor independently 
entropy-coding (e.g., run-length coding) its zero-crossing data 
ignoring the inter-sensor correlation. The compression effi- 
ciency of independent coding depends on the specific un- 
derlying correlation structure. Let pi S [0, 1], « = 1, . . . , iV, 
denote the fraction of snapshots in which the zero-crossings 
occur at the i-th sensor in a Nyquist-interval. Since there is 
exactly one zero-crossing in each Nyquist-interval in each 
snapshot, X^iLi-P* = 1- Sensor i can compress its zero- 
crossing information to the rate of h2{pi) bits per snapshot, 
where h2{p) = -_plog2(p) - (1 - p) \0g2il - p) , p £ [0,1], 
is the binary entropy function [34]. The total bitrate in a 
Nyquist-interval is given by ^2(^1) < Nh2{l/N) where 

the inequality is due to the (strict) concavity of /i2(-) and 
Jensen's inequality [34]. This upper bound will be attained 
if all sensors see the same number of zero-crossings on the 
average. Thus the total number of bits per Nyquist-interval 
per snapshot Rnq{N) for independent entropy-coding is not 
more than N ■ /i2(l/7V) < 1 + logiV bits per snapshot per 
Nyquist-interval. Thus RNgiN) = O(logiV). 



2) Distributed coding of raw data: First, consider the joint 
compression of the raw binary data gathered by all sensors 
in a Nyquist-interval. By design, there is exactly one zero- 
crossing (whose direction is fixed) per Nyquist-interval per 
snapshot. Hence, the A^-length vector of raw binary sensor 
data in any Nyquist-interval can take only N distinct values in 
each snapshot. Hence, even while ignoring the exact statistical 
structure of the temporal evolution, in the worst case, no 
more than log2 (N) bits per snapshot per Nyquist-interval are 
needed to encode these A^-length vectors (being the maximum 
entropy-rate of a source over an alphabet of size N [34]). 
This represents the worst-case joint entropy-rate (whenever the 
entropy rate exists) of all the (binary-valued) sensor outputs in 
any Nyquist-interval. However, there exist distributed source 
codes [33]-[35] which, by exploiting the underlying statistical 
correlation between sensor outputs, can achieve a compression 
efficiency of Rnq{N) = log2(A^) bits per Nyquist-interval 
per snapshot without any sensor collaboration. Further, if the 
joint statistics of all binary sensor data within each Nyquist- 
interval has a symmetric structure, then it is possible to 
noncollaboratively encode the data at each sensor at the evenly 
distributed per-sensor rate of Rsensor{N) = -^\og2(N) bits 
per snapshot. 

B. Information transport to CPU and Scaling Law 

If the processed and coded data from each Nyquist-interval 
from each snapshot can be somehow moved to the CPU 
then the distortion (maximum expected point error) in re- 
constructing each temporal snapshot will goes to zero as 
D = 0{2-^«Q) = O(^). Equivalently, for a fixed D > 0, 
the number of one-bit sensors per Nyquist-interval needed will 
be finite with cx If the field is also temporally bandlim- 
ited and snapshots are taken at a stable temporal Nyquist- 
sampling rate then each temporal slice can be reconstructed 
to a precision O(;^)0 Hence, through temporal interpolation, 
the entire spatio-temporal field, that is, the field value at each 
point and at each time instant, can be reconstructed to a 
point- wise precision of 0(-^). Spatially bandlimited fields, 
however, are not spatially limited: there are an infinite number 
of Nyquist-intervals. A practical way to address this problem 
is to consider how the field reconstruction error in a compact 

'"Every lower-dimensional spatio-temporal slice of a bandlimited field is 
bandlimited to the same set of frequencies. 
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region of interest decays as the field is sampled over a growing 
neighborhood of sensor deployment. 

1 ) Field reconstruction in a compact region of interest: Let 
[—J, j] be the interval of interest with the CPU located at 
the origin. A symmetric interval around the CPU is considered 
only for clarity and is not a restrictive assumption. The field 



L+L' 
A ■ 



L+L' 
A 



We want to 



is sampled in the larger interval 
analyze realizable rates of decay of reconstruction error with 
L'. Over each Nyquist-interval, consider N one-bit sensors 
spread uniformly as in Figure |5] 

Sensors within the region of interest 

^•••poooo o®o oooog«»«f 

CPU 



Data-flow 



Fig. 5. The field is reconstructed over [—L/X,L/X] by sampling over 
a larger region [~{L + L')/\, [L + L')/X\ to meet a desired recon- 
struction quality. Distortion D{N), geographical size jiL + L'{N)), 
and network rate Rnet{N) are inter-related and depend on the number 
of one-bit sensors per Nyquist-length N. Links closer to the CPU carry 
greater (but always finite) traffic. 

For each time snapshot of the field, let T := {ti,l e T} 
be the collection of locations corresponding to the mid-points 
of those inter-sensor intervals where the first sign-change 
is detected in each Nyquist-interval. Here, T := {—L — 
L', . . . ,L -\- L'}. As described in the previous subsection, the 
zero-crossings within each Nyquist-interval, can be located us- 
ing neighbor- to-neighbor local communication or alternatively, 
the raw binary data at each sensor can be compressed using 
a distributed source code. Let Pi{t,T) denote the Lagrange 
interpolation polynomial for the l-th sampling location in T 
[1]. Specifically, Pi{t,T) = n,_^i(i - tjWi - tjT^- Let 



J2{-diU))Pi{t,T) 



lei 



be the Lagrange-interpolated reconstruction of f{t) based on 
the samples at ti e T, where d{t) is the dither field as defined 
in Section |V] Thus /Lag(ii) = —d{ti) for all I. In [11] it was 
shown that Vt S [- 



L L] 
' A ' A J 



\.f{t)-f..M<^ 



2L' + 1 



(7.1) 



if A > ^ . The first error term in ( 17. Il l arises from the finiteness 
of the sampling window and the second error term is due to 
quantization effects. 

If L'{N) is chosen such that 

L'{N) = e(logiV) because A > 
that for this choice of L'{N), 



then 



2\) W 

. From ( 17. It it follows 



D{N) sup 



\.f{t)-fUt)\^o 



N 



(7.2) 



2) Scaling Law for Multihopping: Now suppose that the 
compressed data from each sensor (from a large number 
of snapshots) is routed through neighbor-to-neighbor local 



communication in the direction of the CPU0 The node-id 
(address) information will be part of the header of each data 
packet sent to the CPU but will occupy only a vanishingly 
small fraction of the total data rate for a sufficiently large 
number of snapshots. The focus is on the sustainability of data 
rates with delays taken out of the picture. The total number 
of bits per snapshot generated by the entire network is given 
by 

Rnet{N) 2(L + L'{N))Rnq{N) = 9 ((logiV)^) , (7.3) 

where Rnq{N) = (log A^) is the number of bits per Nyquist- 
interval per snapshot. Eliminating from ( I7.2l i and ( I7.3l l gives 



D{Rnet) — O (i?„et2 



(7.4) 



where (3 



log ^ - o(l). From (|T2li, (O, and dH 



it follows that any nonzero target distortion D > can 
be attained at the CPU provided that N (and L'{N)) are 
sufficiently large (but finite) and the capacity of the busiest 
links (last-hop to the CPU) is of the order of Rnet{N) bits per 
snapshot. The network traffic decreases approximately linearly 
with distance from the CPU. We summarize these results in 
terms of the following realizable scaling law. 

Multihop Scaling Law: With N one-bit sensors per 
Nyquist-interval, 0(logA^) Nyquist-intervals, and total net- 
work bitrate Rnet = ©((logA^)^) (per-sensor bitrate 
8((log A^)/A^)), the maximum pointwise distortion goes to 
zero as L> = 0{{logN)^/N) ox D = O^Rnet^-'^'^^). 

3) Gaussian Multiple Access Uplink: As an alternative to 
multihopping, suppose that a real Gaussian multiple access 
channel, bandlimited to Wc Hertz, can be realized for the 
effective multiuser uplink communication from all the sensors 
over 0(logiV) Nyquist-intervals to the CPU. Then the sum 
capacity, in bits per second, of the uplink channel grows as 
CsumiWcPtot) = Wc log^il + e{PtotiN))) [36], where 
Ptot is the sum power of all Q{N log N) sensors. A suffi- 
cient condition, based on the separation of distributed source 
coding and multiuser channel coding, for successful transport 
of sensor data to the CPU is given by Csum > Rnq x 
snapshots per second [34]. If Wc > 1 and each sensor has a 
sufficiently large (but fixed) power, then Ptot = 8(A^log-/V), 
Csum > RNqiN) X snapshots per second, and successful 
data- transport to the CPU can be achieved with distortion 
decreasing as in il.2i . 

4) Discussion: As N tends to infinity, D approaches zero. 
However, for a fixed, nonzero target distortion D, the number 
of sensors and the network rate needed is always finite. 
This should be contrasted with the following result discussed 
in [9] in an information-theoretic setting: if all sensors use 
identical deterministic scalar-quantizers then even if the sensor 
density approaches infinity, the field reconstruction MSB is 
bounded strictly away from zero. The results in [9] hold 
for all processes for which the probability of crossing some 
quantizer threshold is positive. This includes bounded WSS 
bandlimited processes studied in our work and also possibly 

' ' A detailed description of the pliysical-layer and scheduling aspects of the 
local communication are beyond the scope of this work. 
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those which do not have a sufficient density of quantization 
threshold crossings for the reconstruction error to converge 
to zero as the sensor density increases to infinity [37]. The 
use of identical scalar quantizers can fundamentally limit the 
ability to decrease distortion by increasing sensor density. 
This is clearly seen by considering the extreme example of 
reconstructing a constant field using identical one-bit scalar 
quantizers. This suggests the need for having "diversity" in 
scalar quantization. Dithering is one way of achieving this 
diversity. 

Also, as the desired distortion at the CPU approaches zero, 
one of two things will necessarily have to take place to sustain 
the desired performance: (i) If the CPU has only a finite 
number of links then the capacity of at least one of these links 
must approach infinity. This consequence is fundamental to 
rate-distortion theory and cannot be bypassed because for any 
continuous source and any reasonable distortion criterion, such 
as the space and time averaged mean squared error (MSE), 
zero distortion requires infinite rate, (ii) If the maximum 
capacity of each link is bounded then the number of links 
(independent communication channels) needed to sustain the 
desired level of reconstruction quality must go to infinity. 

C. Extension to two and higher spatial dimensions 



y 




zero-crossings 




Fig. 6. 2-D distributed sampling of spatially bandlimited fields. 

Horizontal dimension densely sampled: N one-bit sensors uniformly 
distributed over each liorizontal Nyquist-interval. Vertical dimension 
sampled at the vertical Nyquist-period 1/A. Arrows indicate flow of 
compressed data towards the CPU. Distortion J, as 0(1/ N) as the 
rate per Nyquist-area t as (log TV) lilce the 1-D case. 

Although our focus has been on 1-D fields, the idea of 
Nyquist space-slices can be used to effect a simple extension 
of results (though by no means the best possible) from the 
1-D one-bit dithered sampling framework (with multihopping) 
to two and higher dimensions. We only explain the essence 
of this idea, using Figure |6] The key observation is that it 
is sufficient to sample only one out of the many orthogonal 
spatial dimensions densely (and uniformly) using one-bit sen- 
sors while the other spatial dimensions are sampled at their 
respective Nyquist-rates. For example, for 2-D spatially ban- 
dlimited fields, one-bit sensors can be deployed along parallel 



horizontal lines where the inter-line spacing is equal to the 
Nyquist-period along the vertical dimension. With N sensors 
spread uniformly over a horizontal Nyquist-length (rather than 
over an area), the number of sensors per Nyquist-area is also 
N. The field along each horizontal line corresponds to a 
horizontal spatial slice of the field and is therefore bandlimited 
to the same set of horizontal frequencies. Applying the results 
of the 1-D case it follows that the maximum reconstruction 
error of each Nyquist space-slice (and hence the entire spatial 
field) is of the order of 0{1/N). The bits per snapshot per 
Nyquist-area grows as (logiV). If the field is also temporally 
bandlimited, snapshots can be taken at the temporal Nyquist- 
rate. 

VIII. Concluding remarks 

The problem of deterministic and stochastic distributed sam- 
pling of bandlimited sensor fields with low-precision ADCs 
was addressed in this work. The possibility of achieving 
any desired reconstruction quality using a multitude of low- 
precision sensors was shown using a dither-based sampling 
scheme. More importantly, the feasibility of flexible tradeoffs 
between the oversampling rate and the ADC-precision with 
respect to achieving exponential accuracy in the number of 
bits per Nyquist-interval was demonstrated. This exposed a 
key underlying conservation of bits principle. An achievable 
information scaling law for field reconstruction was also 
derived. Interestingly, this is possible using only neighbor- 
to-neighbor communication and distributed source coding, 
making it attractive for sensor networks. The extension of 
results from the deterministic to the stochastic setting in a 
strong almost-sure sense was also carried out. This paper 
is but a first step towards understanding the fundamentals 
of distributed sampling theory. Interesting research directions 
include extensions to fields with an unbounded dynamic range, 
e.g., Gaussian random fields, and addressing the effects of 
sensing noise, random deployment, location/timing errors and 
uncertainties, and synchronization and scheduling issues. 
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Appendix I 

Examples of amplitude limited, bandlimited 
stationary processes 

Two examples of stationary processes with uniformly am- 
plitude limited sample paths and autocorrelation functions 
belonging to SL([— ly, VF]) for some > are presented. 
The first one is strictly stationary, the second is WSS. 
Example 1: Let Z{t),t £ M, be any strictly stationary 
process (e.g., any WSS Gaussian process). By passing Z 
through a hard-limiter (a pointwise operation), y{z) := 
sign(z) min(|z|, 1), one obtains a strictly stationary process 
Y{t),t G M, which is amplitude limited. Let X be the 
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process generated by passing Y through any BIBO-stablfl 
linear, shift-invariant (LSI) system, with an impulse response 
belonging to BL{[-W,W]) n C\R). Then X will (i) be 
amplitude limited (in every sample path), being the output of 
a BIBO stable LSI system with an amplitude limited input, (ii) 
stationary, being the output of an LSI system with a stationary 
input, and (iii) have an autocorrelation function belonging to 
BL{[—W, W]), being the output of a bandlimited LSI system. 
Example 2: Let be any discrete, uniformly ampli- 

tude limited, WSS, white process, and ^{t) 
t ^ and V'(O) := L Define a process 



sin(7ri) 



TTf 



for 



X{t):^ lim Y[l]TP{t + l + e), VieM, 

L ^oo ^ — ^ 

where 8 ^ Uniform [—1/2, 1/2] is independent of 
Note that i' is bandUmited, belongs to C'^(R) n £^(]R), and 
suPtgR {E;ez + 01} = C < oo. Using this, it can 
be shown that X is a well defined WSS process, with a 
finite mean, and an autocorrelation function which belongs to 
BL{[—W, W]) and is uniformly amplitude limited. Moreover, 
every sample path of X is also uniformly amplitude limited. 

Appendix II 
Proof of Propositions. II 



h{t) is differentiable at each t, and the derivative h' G 
because h' is uniformly amplitude limited and decays 
as Define f'{t) /r/(''')^'(^ ^ r)dT and note that 
\f{t)\ < A\\h'\\i < A{W + W + 5){W + 5) < oo. Since 
/ e ZBL{W,d) for all 5 > 0, we have < 2AW'^. 

It remains to show that the derivative of f{t) is f'{t). 
To see this, define /i'^(i) := ^ {h{t + e) - h{t)) , e G (0,1] 
and note that \h[{t) — h'{t)\ goes to zero with decreasing 
e for each t G M and can be bounded from above by an 
absolutely integrable function q{t) that does not depend on e. 
Since |i (/(< + e) - /(t)) - < A\\K - h'\\u the result 

follows from Lebesgue's Dominated Convergence Theorem 
[38, Theorem 1.34]. In fact, by using the same technique 
recursively, it can be demonstrated that f{t) is differentiable 
an arbitrary number of times and all derivatives are uniformly 
amplitude limited. ■ 

Appendix III 

Lifting sampling and interpolation results from 
BL{[~W, W]) TO ZBL{W, S) 
For all {t, t) G M^, as i | oo along positive integers, 

L 

Mt^r) := \f{r)mti-T)\\Mt-ti)\ ] 

l^-L 
oo 

u{t,T) := \f{r)\Hti~Tm{t~ti)\. 

1 — — 00 

Since J|/| |oo, 1 1^| |oo < oo and for all {t,L) G M x 
N, j:i=-L\Mt~ti)\<C 

0<Ki,^)< ll/IUI|/i||ooC<oo. 

'^BIBO stands for bounded-input-bounded-output. 



Also, since < oo, for all {t,L) G M x N, 

'^L{t,T)dT < C||/||oo||/j||l <oo. 



To summarize the above in words, ^{t, t) is a nonnega- 
tive, uniformly amplitude limited, measurable field and for 
all {t,T) G R^, VL{t,T) is a nondecreasing sequence of 
nonnegative real numbers converging to v{t, r). For each 
{t, i) G R X N, VL{t, t) is a measurable, absolutely-integrable 
function of r. Hence, by Lebesgue's Monotone Convergence 
Theorem [38, Theorem 1.26], for all t G M, 



r)dr= lim / i'L(t, r)dr < C||/||oo||"-||i- 



Thus, for all t G M, v{t, r) is a measurable, absolutely- 
integrable function of t. For all (t, r, L) G x N let 



9L 



{t,T) :=/(t) J2 h{ti-T)in{t~ti) 



By assumption (ii) in the Lemma and because ||/||oo < oo, 
as L I oo along positive integers, for all {t,T) G M^, 

gLit,T)^ fiT)h{t-T). 

Furthermore, for all {t, t) G R^, 

\gL{t,T)\ < v{t,T), 

that is, for each t G M, ^^(t, r) is dominated by v{t^ r) which 
is an absolutely-integrable function of r. Hence, by Lebesgue's 
Dominated Convergence Theorem [38, Theorem 1.34], 



f{T)h{t-T)&T 



lim / gL{t,T)dT, (IILl) 



for all t G M. Since /(r) is a Zakai-sense bandlimited function 
of r, with associated Toeplitz reproducing kernel /i(t), from 
( 13.1b we have for all t G M, 



f{t) = / f{T)h{t-T)Al 



(111.2) 



and also 



/ 5L(t,T)dr = Mt-U) f fir)hiU~T)dT 

L 



(III.3) 



Hence, from jllLlb . (IIII.2b . and (IIII.3b . we have shown that 
for all t G R, 



fit) 



lim V fiti)Mt~ti). (III.4) 



Since ||/||oo < oo and for all {t,L) G MxN, Ef=-L \i'i{t~ 
ti)\ < C < oo, the series in ( IIII.4b converges absolutely and 
uniformly on all compact subsets of R. This completes the 
proof of the extension of sampling and interpolation results 
from BL{[-W,W]) to ZBL{W,d). ■ 
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Appendix IV 
Design of 6-bit dither fields 

There are several possible approaches for designing a fe-bit 
dither field. Here we present only a simple natural construction 
based on dither fields for one-bit sampling {b = 1). For any 
1 < b < k, let M := 2^~^. For the design, it is sufficient 
to consider the single interval [Ao,i?o] ~ [^i 171 ~ ''']'''' = 
1/(2*^ A), i.e., the interval next to the origin corresponding to 
/ = in Section IVll We define a 6-bit dither field as follows 



(IV. 1) 



where c > (l+7r)/4 is any constant and d{t) is any dither field 
used in the one-bit dithered sampling scheme of Section [V] 
with maximum amplitude 7 > 1 and a derivative whose ampli- 
tude is bounded by A. From Section lV-Al we recall that d{Q) = 
—d{l/X) = ±7. We may assume, without loss of generality, 
that d{0) > so that 4(Ao) -4(Bo) = 2c/A/ > 2/M. 
Also note that < = < if A 

because 1 < 5 < fc. Thus, A := sup < ^^A. 

It will be shown that the dithered-field [/ + db] {t) crosses a 
level in the set {0, ±jj, ... , ±-^7^} in the interval [^0, ^o]- 
By design, db{t) is continuous. The intermediate value theo- 
rem for continuous functions is used to prove the existence of a 
level-crossing. Specifically, it will be shown that [f + db]{Ao) 
and [/ + dbKBo) lie on different sides of some level. Let 
Lj, —M < j < {M — 1), be quantization intervals given by 



~oo, 



.M 

' M- 



1+1 
M 

-1 



M 



00 



for i = -M, 

for - {M - 1) < j < (A/ - 2), 
for J = (A/ - 1). 



therefore, f{Ao) + db{Ao 
i Lj for j < -(A/ - 1). So let [/ 



> -1 + — 



Since 4(^0) > jj. 
Thus, [/ + db]{Ao) 
db](Ao) e for some j, -{M-2) < j < (A/ -2). We now 
show that [/ + df,] (Bq) lies in a different quantization interval 
than Lj. We have the following inequalities 

2c 



[f + db]iBo) 



< 
< 

(Hi) 

< 



fiAo) 

M 
J_ 

A/' 



AT' 
n{Bo - An) 



2c 



2c 
M 



+ 7r(So - Ao) 



2c 
A/' 



where (i) follows from the mean value theorem applied to f{t) 
at the end points Aq and B^ and noting that |/'(i)| is bounded 
by TT (Bernstein's inequality), (ii) is because [/ + db]{Ao) = 
f{Ao) + j§ e L,,^iM-2) < J < (A./-2),^ /(Ao) + |f < 
^^H^ and {in) follows from c > (1 + 7r)/4, A > 1, and some 



M 

algebra. Since [/ - 
some j' < {j — 1), 



-db]{Bo) <j/M, [/ + 
i.e., [f + db]{Bo)iL,. 



db]{Bo) e Lj, for 
This together with 



the assumption that [/ + db]{Ao) e Lj,-{M - 2) < j < 
{M — 2) shows that there is a level-crossing of [/ + db]{t) in 
the interval [^Qj-Bo]- 

In the special case when [f + db]{Ao) E L(7v/-i), then [/ + 
db]{Bo) = f{Bo) - 2c/ M < 1 - 2/A.f. Hence, [/ + 4](So) 



lies in the quantization interval Li,i < [M — 3) and the level 
(A/ — 1)/A/ will be crossed by the dithered field. 
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